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ABSTRACT 



This paper presents descriptive research comparing levels of 
cognitive and reflective development among preservice and inservice teachers. 
A question about the effectiveness of the state educational reform law was 
used to collect data identifying student reflection levels. The question was 
presented as a written assignment to undergraduates in an assessment course 
(taken just prior to student teaching) and graduate students in an 
educational theory course. Data analysis indicated that how respondents made 
use of evidence related to undergraduate/graduate status and school level 
(elementary, middle, or high school) . Results found no significant 
differences between undergraduates and graduates in their knowledge scores. 
There was no correlation between years of teaching and knowledge level 
scores. Half of the inservice and preservice teachers scored at knowledge 
level three, which implied that they would believe what they read or heard 
regarding educational approaches without ascertaining biases, considering 
multiple sources of evidence, or determining context. Regarding justification 
levels, there was a small mean difference between undergraduate and graduate 
students which revealed that scores of graduate students exceeded those of 
undergraduate students by a small measure. Some respondents scored a full 
level higher in justification than in knowledge. (Contains 28 references.) 
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Reflective Decision-Making and Cognitive Development: A Descriptive Study 
Comparing the Reflective Levels of Pre-service and In-service Teachers 

A. Objective 

To present descriptive research comparing levels of cognitive and reflective 
development in pre-service and in-service teachers. To offer instructional strategies 
in order to increase the reflective levels of pre-service and in-service teachers. 

Reflection advocated by many educational leaders as an important process for 
facilitating teachers to engender good classroom practices (Colton & Sparks-Langer, 
1993; Dewey, 1933; Mezirow, 1991; Schon, 1987, 1983; Tom, 1999). Many 
investigators have examined the use of reflection in the teaching and learning 
process (Bennett, 1996; Campoy, 2000; Cooper, 1996; Diss, Buckley & Pfau, 1992; 
Dollase, 1996; Egan, 1996; Fernandez-Balboa & Marshall, 1994; Kouba, 1994; 
Oppewal, 1993; Pultorak, 1996; Riggs & Sandlin, 1996; Stoiber, 1991; Wilson & Ball, 
1996; Wise, Spiegel & Bruning, 1999). Reflective decision making is also 
incorparated in national and state standards. An INTASC (Interstate New Teacher 
Assessment and Support.Consortium) standard states that the teacher is, “a 
reflective practitioner who continually evaluates the effects of his/her choices and 
actions on others (students, parents, and other professionals in the learning 
community) and who actively seeks out opportunities to grow professionally" (Darling- 
Hammond, Wise, & Klein, 1995, p. 187). A Kentucky New Teacher Standard specifies 
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that teachers, “reflect on and evaluate specific teaching/learning situations and/or 
programs” (Kentucky Education Professional Standards Board, 1999). 

Often, written reflections that address teacher standards are the most critical 
component of teachers’ professional portfolios. The researchers have found that in- 
service and pre-service teacher reflections demonstrate a wide range in quality. As a 
result, the researchers wanted to investigate the range of quality in reflection and to 
compare the reflective levels of in-service and pre-serviced teachers in order to gain 
insight about how to improve reflection with instruction. 

B. Theoretical Framework 

A search of theories to explain different levels of reflection uncovered a long 
and distinguished history of cognitive development theory. Piaget (1965), identified 
two stages of moral judgment by observing children’s games and by asking 
questions about the consequences of broken dishes. Piaget’s work was advanced 
by Kohlberg (1981) who utilized Piaget’s methodology of telling stories and asking 
questions. Kohlberg expanded Piaget’s two stages into six that described how 
individuals develop moral thinking during childhood and adolescence. Kohlberg 
suggested that these stages were culturally and historically universal and asserted 
that individuals will progress through stages of moral development, as long as they 
are provided proper environmental support. Gilligan (1982) took issue with 
Kohlberg’s stages claiming they were biased in favor of culturally masculine models 
of justice. Gilligan challenged Kohlberg’s “justice orientation” of morality with her own 
“care orientation” of morality. Other researchers have noted that the underlying 
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concepts of the two approaches are similar and they have called for an integrative 
approach that includes both perspectives (King & Kitchener, 1994). 

Perry (1981), perplexed by student’s contradictory perceptions of his courses 
and teaching and was prompted to investigate his students’ cognitive development. 
Perry determined that students evaluated his courses based on their understanding 
of the nature of knowledge. He made specific connections between students’ 
responses to college assignments and their ethical and cognitive developmental 
levels and developed a model comprising nine cognitive and ethical stages. King 
and Kitchener (1994) also targeted an adult population and expanded Perry’s model 
of knowledge development to include how students utilize evidence to support their 
opinions. The King and Kitchener model of reflective judgment includes both 
concepts of justification and views of knowledge forming seven developmental 
stages. 

C. Method of Inquiry 

This study used the King and Kitchener model (1994) to assess pre-service 
and in-service teacher’s levels of reflective knowledge and justification. King and 
Kitchener (1994), Perry (1981), and Gilligan (1982) all used interviews to determine 
participants’ reflective levels. Rather than interviews, the current researchers wanted 
to investigate a more practical way, specifically, a written assignment, to determine if 
they could identify student’s level of reflective judgment. Like King and Kitchener, the 
current research used ill-structured problems to determine reflective judgment. In 
addition, the topic had to be familiar to a wide range of individuals. A question about 
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the effectiveness of the state educational reform law was selected as the means to 
collect data in order to identify levels of student reflection. This question was 
determined to be highly appropriate for eliciting a thoughtful responses since pre- 
service and in-service teachers were likely to be knowledgeable about and have 
opinions concerning high stakes assessment polices. The question was presented 
as a written assignment to undergraduates in an assessment course and to 
graduate students in an educational theory course. A rubric based on the King and 
Kitchener model of reflective judgment was developed to score student responses 
(Figure 1). 

For the undergraduate respondents, the assessment course was taken 
immediately before the student teaching semester. The graduate respondents were 
at various stages in their program. All respondents maintained a GPA of 2.5 or 
higher. The assignments were presented by full-time program faculty who used the 
same directions for explaining the assignment and the same rubric for scoring the 
results. The assignments were blind scored by the course instructors who attained 
an inter-rater reliability exceeding 90%. 

Figure 1 Reflective Judgment Rubric 
Knowledge Levels 

1. What I have seen is true (concrete view of knowledge, knowledge exists absolutely, 
and knowledge is not understood as an abstraction) 

2. There is a true reality that can be known with certainty, but it may not be known by 
everyone. Experts “know” the true reality. 
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3. Authorities may not know now, but they will know in the future. Knowledge is 
certain. 

4. One can not know with certainty. Knowing is idiosyncratic to the individual since 
situational variables may force ambiguity. 

5. There are different ways of knowing. Knowing is contextual and subjective because 
it is filtered through one’s perceptions. The beholder may recognize different ways of 
knowing but can not integrate them. 

6. Knowledge is constructed on the basis of information gathered from a variety of 
sources. 

7. Knowledge is the outcome of a process of inquiry in which solutions are 
constructed from a variety of integrated sources. 

Justification Levels 

1. Justification is not needed or not relevant (beliefs need no justification; there is 
absolute correspondence between what is believed to be true and what is true). 

2. Beliefs are unexamined and unjustified, or justified by an authority figure. 

3. If authorities do not yet know, the thinker will use their own feelings or beliefs as 
justification. 

4. Beliefs are justified by evidence, but the arguments and choice of evidence are 
idiosyncratic. The thinker does not acknowledge qualitative differences between 
experts and non-experts, and may dismiss authorities as biased. 
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5. Beliefs are justified by rules of inquiry for that context. The thinker realizes that 
different rules of justification exist for different contexts, but can not integrate them into 
a structure. 

6. Beliefs are compared based on evidence and opinions from different perspectives, 
and solutions are constructed that are evaluated by multiple criteria. 

7. Beliefs are justified probabilistically on the basis of a variety of interpretive 
considerations, such as the weight of evidence, the explanatory value of the 
interpretation, the risk of erroneous conclusions, the consequences of alternative 
judgments, and the understanding of an issue on the basis of available evidence. 

D. Results and Conclusions 

T-tests for two groups and one-way ANOVA analysis for multiple groups were 
used to test effects of respondent characteristics with respect to reflective judgment 
scores. The Pearson correlation statistic was calculated to explore bivariate 
correlations between respondent characteristics and reflective judgment scores. 

The findings identified factors that appear associated with how the 
respondents make use of evidence to make decisions about their knowledge. The 
research results suggest that how the respondents make use of evidence may be 
associated with under/graduate/graduate status and with schools levels (elementary, 
middle school, high school). 
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Justification Levels (Use of Evidence) 

The ability of undergraduate and graduate students to apply evidence was 
measured using the reflective judgment rubric, which included seven justification 
levels from 1 .0 to 7.0. A score of 7.0 suggests use of a complex cognitive system for 
evaluating evidence. An independent samples t-test analysis indicated a statistically 
significant difference (p>.05) between the justification score means for graduate 
students (mean=3.82, n=59) and undergraduate students (mean=3.68, n=110). 
Approximately 60% of the graduate students achieved a score of 4.0 compared to 
about 40% of the undergraduate students. 

The ability of undergraduate and graduate students to apply evidence was 
analyzed for differences by school level including elementary (n=85), middle school 
(n=32), and high school (n=52). Each of the three groups included undergraduate 
student pre-service teachers and graduate student in-service teachers. A one-way 
ANOVA analysis identified a statistically significant difference for high school level 
scores (mean=3.90) which exceeded both elementary (mean=3.66) and middle 
school (mean=3.62) group scores. 

Respondent’s justification scores had low positive correlations with age and 
the number of years teaching. A Pearson correlation coefficient (r=.25) was 
determined for the relationship between graduate student justification scores and 
number of years teaching. This group, which had taught for an average of 4.5 years, 
held a mean score of 3.82 on the justification measure. When all respondent ages 
were compared to justification scores, a Person correlation coefficient (r=.32) was 
calculated. 
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Knowledge Levels (Nature of Knowledge) 

The research did not reveal differences among the groups with respect to how 
the respondents think about the nature of knowledge. The ability of the undergraduate 
and graduate students to view knowledge as uncertain and abstract was measured 
using a knowledge rubric with seven knowledge levels, ranging from 1.0 to 7.0. A 
score of 7.0 suggests an advanced understanding of knowledge. The respondent’s 
knowledge score did not appear linked to (a) undergraduate or graduate status, (b) 
school level (elementary, middle school, high school), (c) age, or (d) number of years 
of teaching. An independent samples t-test analysis did not uncover a statistically 
significant difference (p<.05) between undergraduate and graduate student 
knowledge scores. Approximately 50% of the students in each group achieved a 
knowledge score of 4.0. 

Knowledge level mean scores were compared for students in the elementary, 
middle school, and high school groups. A one-way ANOVA test did not reveal 
statistically significant differences (p<.05) among the groups. The high school group, 
with a knowledge mean score of 3.90, led the middle school (mean=3.62) and 
elementary (mean=3.66) groups. 

Relationships between knowledge scores and factors including number of 
years of teaching and age were explored using a Pearson correlation statistic. 
Statistically significant relationships were not identified. 

The relationship between respondent’s knowledge and justification scores 
was examined. Justification scores exceeded knowledge scores in each of the three 
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school-level groups. A Pearson correlation coefficient (r=.27) was determined when 
knowledge scores and justification scores were compared. 

E. Educational Significance 

Regarding the methodology of the investigation, findings indicate that a writing 
assignment is an effective method for determining respondents’ reflective levels. To 
check responses, selected respondents were interviewed to confirm the opinions 
expressed in the written assignment. All respondents verbally confirmed their written 
opinions. The ability to readily obtain information about the development of students’ 
reflection levels has important instructional implications for the researchers and for 
teacher preparation programs. 

Regarding knowledge levels, the finding of no significant difference between 
graduate and undergraduate respondents in their knowledge scores was surprising 
as was the lack of correlation between years of teaching and knowledge level scores. 
Surprising, when it might be expected that knowledge level scores would increase 
with time and experience in the classroom. These findings are supported by Harvey 
(1970) who determined that the abstract cognitive levels of college sophomores were 
higher compared to those of classroom teachers. The finding that 50% of in-service 
and pre-service teachers scored at knowledge level three was also perplexing. 
According to studies by King and Kitchener (1994), high school seniors score at a 
knowledge level of three. The practical implication is that level three teachers will 
believe what they read or hear regarding educational approaches without ascertaining 
biases, considering multiple sources of evidence, or determining context. This 
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stagnated development seems to indicate that there is little intellectual stimulation in 
schools to encourage teachers to higher levels of reflection and intellectual growth. 
What should teacher preparation programs do to develop reflective teachers? 

Regarding justification levels, the study found a small mean difference 
between graduate and undergraduate students that revealed that scores for graduate 
students exceeded undergraduate students by a small measure. The study also 
revealed that high school education majors had higher justification scores than 
elementary and middle school education majors. This may indicate that justification 
can more easily be improved through educational experiences than can knowledge 
levels. Also, it is unclear if programs with more rigorous content requirements better 
prepare students to support and defend their ideas or if students with higher levels of 
reflection are attracted to more rigorous programs. 

Acknowledging the modest positive result for graduate students, it was still 
disconcerting that 40% of graduate students produced an assignment with 
unsupported opinions, indicative of level 3 justification. In King and Kitchener’s 
(1994) studies, the graduate student mean score was a level five. As an implication 
for instruction, these results indicate that both undergraduate and graduate students 
need intensive experiences in defending educational approaches and their own 
educational beliefs with respect to biases, multiple perspectives, and context. 

Finally, one finding was particularly intriguing to the researchers. While the 
King and Kitchener (1994) model describe knowledge and justification separately, in 
reporting results, respondents received one score implying that knowledge and 
justification have parallel development. In the current study, researchers were 
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surprised to discover that some respondents scored a full level higher in justification 
than in knowledge. Specifically, some respondents scored a four in their level of 
justification but scored a three in knowledge level. The reverse was never found. This 
suggests that justification may precede knowledge development in some individuals. 
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